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ABSTRACT 

This paper examines the nature of dictionaries and 
encyclopedias, focusing on some of the preparation that has gone into 
the construction of a dictionary of language testing at the 
University of Melbourne (Australia). It discusses the purpose of such 
dictionaries, the nature and size of dictionary/encyclopedia entries, 
and the readability of entries. It also reports on an experiment to 
determine the proper length and difficulty level of possible entries. 
Twenty-one M.A. students were asked to read three sets of entries and 
comment on their length, difficulty, and lexical density. It was 
found that there was substantial agreement among the students as to 
whether several entries were too long, too short, or about right, and 
if they were difficult, easy, or about right. An appendix contains 
copies of the possible dictionary entries. (MDM) 
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, He |bl , und ertake. to compile . Dictionary undertakes that which, if il comprehends the full extent of 
ki ,d«.ig-.l«eknow.himse| f u„,bleK,perform- (JohnsoB 1773) 



C S Lewis (1960) comments on the term 'simple' thus: 'Every impound, 
„rso we hope, can in principle be resolved into simple ingredients, ingredients 
licb are internally homogeneous. And as the compound is a compound, so these 
Xmate ingredients are simples.' (166). 'What is simple or plain is the reverse of 
Indicated. A complicated process is hard to learn and a complicated argument 
Sto follow. Therefore simple comes to mean 'easy'. The idea that .1 1 IS within 
At capacity of those who are simple (in the sense 'unskilled') may perhaps have 
helped this development.' (174). '1 describe the final state of the word as a 
wnantic sediment What effectively remains is not this or that precise sense but a 
geneial appcalingncss or disarmingness.' (179). 

Disarming indeed! Does this mean that there is no objective judgement as 
to what is simple or difficult, that it is all a question of appealingness, a kind of 
political correctness (P.C.) of the lexicon? 

As far as texts are concerned there has been a great deal of investigation of 
what makes for ease of readability. Jeanne Chall (1984) summarises the findings of 
this work, thus: 

'What makes text easy or hard to read and comprehend? The research in 
readability has uncovered over one hundred factors related to difficulty - such 
factors as vocabulary, sentences, ideas, concepts, text organization, content 
abstractness, appeal, format and illustrations. Of these factors, the two found 
consistently to be most strongly associated with comprehcnsibility are vocabulary 
difficulty and sentence length. Various forms of these two factors are included in 
most of the currently used readability formulas. The strongest factor of the two is 
vocabulary difficulty - measured either by a count of unfamiliar words, hard words, 
words of low frequency, words of three or more syllables, or words of 7 letters or 
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|„ a formula, another adds httk to the P^^"^ 0 { difficuUy ln readability 
second strongest and second ^f^J^^^olm^^^ " nd 
formulas. It is very highly J a formula . It is also 

therefore only one sentence factor s usuaHy % 

substantially associated with v ^ buta * y ofwritten text to a high degree 

factor together ^1^^^ 0 7 » °' 9 '"^ 

•t..< .h P «. are other factors that contribute to 
Chall does not of course deny « ™ focus on thc , wo she 

text readability but for the purposes of his paper we w IT ^ ^ 
details, vocabulary and sentence length, althougn 
readability formulas. 

,i„. nrmaration of information for a dictionary (or 
Our concern is with the P rc P arat,on 01 , f thc d i st i nc tion). 

m ore specificaUy for an e«cydopedu^»»^ "£ZL construction of a 
Tbe paper reports some of the prepay on that h as gP»e 
Dictionary of Language Testing (Davies 1992) 
Centre, University of Melbourne. 

MaKing .formation avaiUble to ^insUtut^di - - Jj^ 
data to inform, the heart «*^'JZ^£2» L pedagogic de.ivery of 
Or to put it another ^^J^T^Mt difference existing in nature 
information. Not simple, feat is an issue oi P ^ ^ |s nQ 

whereby for example an X is more dlff ^" ^ Ai'^ continuum in nature 
limitation here as to context; in other ^^^f^ y always be a relative 

The history of readability research records 
Coser the text and the reader. *- 
their interaction so that measures "^^t the cloze technique (1957) 
readers are both necessary. Taylo s dev bpm t ^ which 

was a deliberate attempt to combine 4)> focusing 0 „ 

could be used to ascertain readability of »«* s P a P .„ wh \ ch an ori ginal text 

the second language situation, reports on a ' «JJJJ^ comprchc „sion by a 
and its simp.if.ed version ^T^^L^ *•« 
a - at^2^*- -e hypoUiesis 
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wis supported. In that experiment the measure of comprehension was verbatim 
doze* 

In our current task of writing a dictionary, care must obviously be taken 
that the definition or explanations can be understood by the reader. That means 
targeting the dictionary carefully so that the explanations have appropriate 
readability for their audience. Dictionaries therefore have a built-in pedagogic 
function. This raises the questions of just what a dictionary is and in particular what 
sort of word-book is needed for a professional-academic audience. Some views by 
dictionary makers will be of interest: 

Abercrombie N., Hill S., Turner B S (1984) claim that 'A dictionary of 
sociology is not just a collection of definitions, but inevitably a statement of what 
the discipline is. It is also prescriptive in suggesting lines of development and 
consolidation. The problem of definition in a subject as diverse and dynamic as 
sociology is to strike a balance between an existing consensus, however fragile and 
temporary, and a developing potential. The unifying theme of this dictionary is our 
conviction that sociology is an autonomous, elaborated and vital discipline within 
the social science corpus. Our enthusiasm for the subject was sustained rather than 
diminished by the experience of seeking precision within the conflicting range of 
perspectives that constitute modern sociology.' (p. vii). 4 A statement of what the 
discipline is*: a tall order indeed but nevertheless inevitably what all dictionary 
making assumes in its normative role. 

West and Endicott (1935/59) maintain This English Dictionaiy is written 
specially for the foreigner. It explains to him in words 'which he knows the 
meaning of words and idioms which he does not know.' (p. iii). In words which 
s/he knows the meaning of, stresses the welcomed linguistic straitjacket of 
dictionary making. 

Angeles (1981) states that his dictionary Ms intended as an at-hand 
reference for students, laypersons, and teachers. It can be used as a supplement to 
texts and philosophy readings; it can also be consulted for philosophy's own 
enjoyment and enlightenment.' (p. ix). Even the 'laypersons' Angeles refers to 
must surely be informed, interested, educated and so on. Audience is critical and 
when it includes students necessarily demands some measure of simplification, if 
S not of language, certainly of substance. 
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The idea for writing a Dictionary of Language Testing by colleagues m the 
NLL1A Language Testing Centre, University of Melbourne arose out of several 
neXhcneed for a kind of in-house set of glosses so that we .11 know what we are 
talking about, our own register, then, as with text-books, the necessary compromise 
between the profession (those like ourselves working in Applied L.ngmst.cs and 
language testing) and the public. Such a compromise targets those with general 
,ther than specialist knowledge, ie MA students of the relevant "Kaphnes. Very 
much, in fact, like the audience targeted by Richards, Piatt and Weber .n the.r 
Longman Dictionary of Applied Linguistics (1985). We have found that working 
together on this dictionary is a felicitous way of sharing and educatmg one another, 
precisely because it defines, explores and creates the very register we need for our 
work Those working on the dictionary (Alan Dav.es, T.m McNamara, Cath.e 
Elder, Annie Brown, Tom Lumley, Chris Corbel, Yap Soon Hock), are .11 very 
much p.rt-time and we are aware that this is a long term task. The Longman 
Dictionary of Applied Linguistics, after all, was 4 years .n the making. A larger 
contributing group of authors provides wide coveragt but needs marshalling and 
organising. And in such an exercise organisation is essential. 

As I have already suggested the problems have to do with audience and 
definition; but equally important are selection and coverage, scope and format of 
entry. In attempting to reach agreement over these matters we have been helped by 
the realisation that such concerns are not at all new in lexicography. If mdeed what 
we are doing is lexicography. 

Let me quote from Kipfer's Workbook (1984:1) 'A dictionary is a 
reference book containing the words of a language or language variety, usually 
alphabetically arranged, with information on their forms, pronunc.at.ons, functions 
meanings and idiomatic uses. A dictionary may be more than a reference book 
about words; it can contain biographical and geographical knowledge as well as 
lists of colleges and universities, weights and measures, and symbols; and the 
introduction may include articles about aspects of language and dictionary mak.ng 
The entries themselves may contain not only pronunciations and mean.ngs, but 
information about grammar and usage and even the kind of information an 
encyclopedia gives about the thing the word names.' 

There are indeed many terms for our type of ambition: are we writing a 
dictionary, an encyclopedia, a word list, a glossary, a reference list? Op.tz writes of 
, 'segmental dictionary', but is that what it is or a glossary, perhaps -.e a l.st o 
technical terms rather than an attempt 'to isolate a distinct reg.ster wh.ch is what 
Opitz means by a segmental dictionary. (Opitz 1983: 58). 
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But is it a glossary? Hartmann defines glossary as a 'word-list with 
explanation of meanings' (Hartmann 1983: 223). Moulin describes a glossary as a 
list of glosses appended to text, often specialised, and details two techniques of 
ordering, by areas of interest and by alpbcbet: 'most authors (of specialist 
dictionaries) are neither linguists nor professional lexicographers, but specialists in 
the particular discipline... these glossaries are commissioned. ..to try and introduce a 
measure of normalisation in the use of specialist terms and thus facilitate the 
exchange of information' (Moulin 1983: 146). We will return to that concern for a 
'measure of normalisation'. 

Is it an encyclopedia? Hartmann tells us that encyclopedic information has 
to do with 'practical knowledge of things versus lexical information' (Hartmann 
1983: 223). A more elaborate distinction is made by Read (1976: 713ff) quoted in 
McArthur 1986: 'The distinction between a dictionary and an encyclopedia is easy 
to state but difficult to carry out in a practical way: a dictionary explains words, 
whereas an encyclopedia explains things. Because words achieve their usefulness 
by referring to things, however, h is difficult to construct a dictionary without 
considerable attention to the objects and abstractions designated'. 

McArthur reminds us that the Encyclopedia Britannica had its origin in 
Edinburgh. Notice its original title: 'The Encyclopedia Britannica or a Dictionary 
of Arts and Sciences, compiled upon a New Plan' (Edinburgh 1768-71, sponsored 
by the Society of Gentlemen in Scotland). The Britannica was a very obvious 
product (no doubt influenced by the French philosophers) of the Scottish 
Enlightenment, that high point in Scottish history, when Scotland truly was the 
clever country. From that high point we are brought down to earth by the comment 
of William Smellie, one of the original authors: 'with pastepot and scissors I 
composed it' (W. Smellie in Kogan 1956: 14, quoted in McArthur: 106-7). 

McArthur suggests as a way of resolving the overlap in the uses of the 
terms Dictionary and Encyclopedia that it is probably best not to bother. 'The 
simplest way' he says 'of resolving the tension seems to be to accept the way in 
which the early encyclopedists handled the matter. In this dilemma we in fact work 
along a continuum rather than within separate containers, where one extreme is 
words and words alone, and the other is referents and referents alone' (McArthur 
1986: 104). At one end of McArthur's continuum is the dictionary, at the other the 
encyclopedia and in between the encyclopedic dictionary. 

McArthur suggests as a way of relating dictionaries and encyclopedias 
(which b the USA and France, but not in the UK, have, he says always been linked) 
the following pair of termr 'thai could be useful in studying the world of reference 
materials: 
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, • n* raobv whicb deals with the world of words and the 
!. roicro-lcxicography> wnicn alphabetic dictionary)- 

1986: 109). 

ta own products are the Longma ^ ^iX,oductio« to their Longman 
Dictionary of Linguist.cs and ™onet.cs. in >sk wfao „ ^ 

Dictionary of Applied Linguistics, totanfc fo J sludcnt s of Applied 

dictionary intended for both in training and in the 

Linguistics and General Linguistics, iang s 

Geld. (1985: v). 

^..^ clear and simple definitions which 
•Our aim' they say 'has been to I^^VJ^,ech«tal language, 
communicate the basic and essenUa ^ZL^ccs show links to 

Definitions are self-contained as ^^ We infom ,tion where a fuller 

Mo-elpfultoushiourg^^^ 
Preface to his AJEilSLJ2idl0^^ for lhis book is 

remain doubtful even now whe her j » mo* PP lhemsclv es were less 
dictionary-. The defi.it £nal^ «Uof * -Ui y ^ ^ 
Uluminating than one might have J" J |Ul sc veral illustrations, to 

introduce in addition a ^ , CCO rdingly contain encyclopedic 

capture the significance of a term. Mort « ^ which , lcrm was used, 

obligator^ 

NoAave 1 made use of the ^ry-nsL open a dictionary 

preferred to work on the inc*** " "^id be given a satisfactory accoun 
with a single P^^^^^it^ ^^ unde 
of that term as immediately PERFORMANCE, and so on. As a 
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versa. This repetition would be a weakness, if the book were read from cover to 
cover; but a dictionary should not be used as a text-book, and while the result has 
been a somewhat longer volume than would have been the case if the 'See...* 
convention had been used, I remain convinced of the greater benefits of ook-up 
convenience and entry coherence/ (p. 5). 

After some preliminary trials, pilot entry writing and a small-scale survey 
of the entries among teachers and MA students, we determined on the following 
guidelines for ourselves: 

I the entries should be on the encyclopedia side of McArthur's continuum, 
m ore than language-definitional, explaining where appropriate; 

2. they should where possible (and appropriate) give examples so as to situate 
the explanation; 

3. they should accept overlap, in Crystal's sense, so that referring to other 
entries for necessary explanation would be avoided, except where necessary for 
informative purposes; citations would be minimised except in the sense of the 
informative.purposc above; 

where possible one clear definition should be attempted, in other words coming 
down on the side of being normative rather than descriptive. We have taken the 
view that unlike a truly descriptive dictionary (such as the OED) it is our role to 
contain and confine to 'try and introduce a measure of normalisation in the use of 
specialist terms and thus facilitate the exchange of information.* (Moulin 1983: 
146). 

Whether what we are doing therefore should be called a dictionary or an 
encyclopedia is really beside the point. But while it does veer towards the 
encyclopedia side of the McArthur continuum it retains important aspects of 
dictionary-ness. It does attempt definitions, it avoids essays (so it is not a Glossary 
either: 'I have retained the procedure of organizing the Glossary as a series of 
essays' (Abrams 1981: v) but unlike many dictionaries it has no information of a 
pronunciation kind (though obviously it would not eschew this where it seemed 
relevant) nor does it systematically contain historical material about derivations. So 
it probably is what McArthur calls an encyclopedic dictionary. 

We are in our Language Testing Dictionary concerned to establish a 
uniform style of entry and at the same time to ensure adequate coverage. To 
illustrate these questions and through them the importance in our view of being 
more encyclopedic than dictionary-like, I turn now to a comparison of alternative 

entries. 
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the choice between the A version and the B vers on beU,w ,jm ^ § 
ro uch in favour of the B ^^^iTJon is much shorter than the B 
,,rger number of entr.es. In each ca* A like .no the A 

version, in some sense therefore ^^"°^ com ^ tison , reported below our 
version more -"J^^J^^ nature of this Diction.ry the B 

Experiment and Results 

, . ,xi = ?n were asked to read three sets of entries 
A clsss of MA students (N - = ) were «s ^ ^ ^ short Qf 

(see Appendix) and comment or. {D difficult , too essy or .bout right, 

ibout right; .nd (2) their «J^2*««y choice, to h.ve to make. 
With hindsight it is apparent ^ ^«ewc^ J ^ providc Wlth 

of their lexicl density (H.llid.y 1985). J*?^^ " rcports th.t in inform.1 
o lexicl to gr.mm.tid lo.d ing M ^"2,.^ it is typiclly 
.poken English lexical dens.ty » J^ 1 *^^ * can be as high as 10-13 

„,« .„ «,» — < -i— - *■ »«— * "" ,,e " ,s ,,on8s,de *• 

lexical density finding for each entry. 
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Draft Entries 





1A 


IB 


2A 


2B 


3A 1 


3A 

2 


3B 


Too long 


0 


9 


0 


7 


0 


0 


16 


Too short 


14 


0 


17 


0 


19 


8 


0 


About right 


7 


12 


3 


14 


2 


13 


4 


Difficult 


5 


10 


6 


3 


6 


2 


9 


Easy 


7 


1 


9 


0 


13 


4 


1 


About right 


9 


10 


4 


18 


2 


14 


9 


Lex Density 


7 


7 


12 


5.8 


11 


6.5 


6.8 



Table 1, Draft Entries : Responses and l e xical Densjty 



As a rule of thumb we suggest that 50+% approval for an entry indicates 
acceptability. On that basis, two entries, 2B and 3A 2 may be labelled acceptable. 
That judgement is supported by the lexical density comparison for these two entries. 
Note that 2B has a much lower lexical density than 2A (less than ball) and that 3A , 
while much lower on lexical density than 3A,, is marginally lower than 3B. The 
problem with 3B, which we had predicted would be rated as more acceptable by the 
class, seems to be sheer length. It is interesting that (see Table) there is substantia] 
agreement (16/21) that this entry is too long and yet at the same time as many as 
9/21 accorded it 'about right' for difficulty. A similar result emerges for Entry 1 
where there is no separation between the A and the B versions in terms of difficulty 
(both entries obtain a lexical density result of 7). At the same time (and here is the 
similarity with entry 3B), there were 9 responses in the 'about right' response for 
1A and 10 for IB. We might therefore suggest that acceptability as indicated by 
being accorded 'about right' for difficulty is in part a function of lexical density. 
Where lexical density docs not discriminate (as in the 1A and IB entries) neither 
choice is regarded as being acceptable. Where there may be little to choose in terms 
of lexical density (as between Entries 3A 2 and 3B) length of an entry may militate 
against the choice of an entry (as with Entry 3b). 

Of course it may be queried whether a response of 'about right* is 
appropriate, whether indeed (as with the figure quoted above for scientific writing, a 
lexical density of between 10 and 13) specialists tolerate a high density, a 'more 
difficult' entry. But that is after all a comment on the sampling of our responses 
and it ts indeed our contention that the class whose responses wc report here are the 
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appropriate audience for our Dictionary of Language Testing, students on Masters 
ZJs who are in the process of being introduced to classes in Apphed Linguae* 
tarftftag Language Testing. Specialists in the field (pace Abercromb.e et al 1984) 
^re not our concern in this task; to what extent a dictionary for speaahsts s a vub e 
activhy we remain unclear about. In our view dictionaries (and here we would 
agree with Abercrombie et al 1984) are always normative in the sense that they are 
drawing boundaries with a pedagogic intent. No dictionary certa.nly not a 
segmental dictionary, can ever satisfy the specialist! For that » wh., wek ave 
decided to term our effusion, a segmental dictionary: as such .t ret .ns 
p"fessional/vocational/registral association and at the same .me iU 
normative/pedagogical purpose which is what makes our attempts at s.mphficanon 
necessary. 

. m tt0 , «, k*l in .ex.cograpby as to forge, «ha, words .re ,he daughter, of earth, and .ha. Ihing. 
(including deed.) are the .onsof heaven' (SamueUohnson> the Preface of hU Denary 1755) 
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APPENDIX 

STANDARD DEVIATION. 

EnttX-lB , f t u. ni«?PFRSION of a SAMPLE which can be 

52d£ . statistical measure of the J^™'^^ DEVIATION, that 
^d i„ a sundardised square tooi .torn* ^ANDARD ^ 

is to say that the variance of a " ™ be compared with its 

dispersion of a sample on one mcas "" f ^ r c ^; s m J sharcd Jriance. Such a 
dis^rsion on another; this companson JJ^"^ J r furthcr comparisons 
comparison is achieved by means of a l"™*"' * f ANALYS IS OF 

VARIANCE. 1 be square 01 iuc correlate 0.7 would have a 

overlap between the two tests. 

— causality T Z ^r^™ 
A»ov. la commonly available oo computer .laUstol p«k.6<* »JJ« ^ 
.1 ,hc cod of an iolcnocdi... l.ogoagc coo.se ts sbown .0 be >«« to" «y ™*> 

.nalysis, what remains for entry scores is trivial, (sec. variance, 
criterion, variable, predict). 
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gias: Systematic cnor associated with any type of group membership, sex and age 
group membership included.' (Jensen 1980). 

Entry 3A, 

Jjja*: 'systematic differences in test performance that are the result of differences in 
individual characteristics, other than the ability being tested, of test takers.* 
(Bachman 1990: 271). 

gptrv 3B 

Bili: bias is defined by Jensen (1980) as 'systematic error associated with any type 
of group membership, sex and age group membership included*. The terms 
systematic error and group are important in this definition. In a trivial way all tests 
are biased against individuals who lack knowledge or skill. But since that is what 
tests are designed to do, such 'bias* or, better, discrimination is not systematic error, 
that is to say there will be random error as in all measurement but it is not 
systematic or deliberate. The group issue is more problematic in situations of norm 
conflict such as recent migrant communities. Should children from such 
communities with only a few years of schooling in the target language (eg English) 
take the same language tests as first language speaking children? In conflict arc (1) 
the general educational norms and standards of the host community and (2) what it 
is reasonable to expect in terms of English language proficiency of the migrant 
children. In situations where the first consideration weighs more heavily, migrant 
children will take the same English test as first language children. In situations 
where the second consideration is more important, a more specialised test of ESL 
may be used. 
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